Neural Subgoal Generation using Backpropagation

Building a world model takes exponential computational costs with the number of obstacles. In real world applications are usually many obstacles, possibly changing their positions over time. In order to cope with a changing environment, a solution has to be adaptive. In order to plan complex trajectories, a system that plans hierarchically shows many advantages. In this article we report on results with a neural (therefore inherently adaptive) subgoal generation system. We show that meaningful subgoals can be produced for two joint manipulators in an environment with obstacles. Unlike many other approaches our approach works (once trained) fast and remains adaptive. Trajectory generation for manipulators is a diicult problem, up to now not satisfyingly solved. Of course there are several classical algorithms that are able to construct collision free paths (e.g. 4]) using only very limited computational resources. Normally these algorithms are based on world models. The construction of the world model takes, in contrast to quick path nding, exponential costs with the number of obstacles to be avoided. Thus these algorithms do not provide feasible solutions for real world problems. In particular such algorithms forbid by construction path planning in a changing environment. An approach that does not need a world model was introduced in 1]. In order to yield a path between two points in joint space, the system tries to connect them using a straight trajectory (with a mechanism that allows to slide along obstacles). If no straight trajectory can be found, the system successively generates random subgoals. It tries to nd a path using all sub-trajectories between the generated subgoals, start, and goal until a valid combination is found. Although this search is stochastically complete, the approach suuers in practice from several disadvantages: 1. the number of subgoals that are tested must be limited (search no longer complete) 2. the system does in no way optimize the generated trajectories 3. the system does not use knowledge available from prior search We construct a system that takes the advantages from 1] but avoids the disadvantages. Following 1] we plan hierarchically by combining straight trajectories that connect start and goal via subgoals. Opposite to 1] subgoals are not randomly generated but by using neural networks. This provides the advantage that usually the rst generated subgoal can be used. Furthermore neural networks ooer the possibility to adapt to a slowly changing environment through constant retraining. In addition to that …